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Abstract 


The  Probability  of  Identification  ( Pjd )  metric  is  a  method  for  measuring  and  scoring  the  quality  of 
maritime  Intelligence  Surveillance  and  Reconnaissance  (ISR)  activities  in  an  operational  context. 
The  metric  presents  a  clear,  easy  to  interpret,  quantitative  measure  of  effectiveness  of  maritime  ISR 
activities  over  a  defined  period  of  time,  and  has  been  adopted  by  Canada’s  regional  operational 
commands  as  one  of  the  standard  metrics  for  maritime  ISR  reporting.  The  aim  of  this  paper  is 
to  describe  the  theoretical  and  technical  underpinnings  of  the  P/[>  metric  for  a  scientific  audience. 
The  current  metric  serves  as  a  starting  point  for  evolving  a  much  more  comprehensive  metric  for 
maritime  ISR. 


Resume 


La  mesure  de  la  probability  d'identification  (P/d)  est  un  mode  de  mesure  et  de  pointage  de  la  qua- 
lite  des  activites  de  renseignement,  de  surveillance  et  de  reconnaissance  (RSR)  maritimes  dans  un 
contexte  operationnel.  La  methode  presente  une  mesure  facile  a  interpreter,  claire  et  quantitative 
de  l’efficacite  des  activites  RSR  maritimes  pendant  une  periode  definie.  Elle  a  ete  adoptee  par  les 
commandements  operationnels  regionaux  du  Canada  cornme  V  une  des  mesures  normalisees  du  si- 
gnalement  RSR  maritime.  Le  present  document  vise  a  decrire  les  piliers  theoriques  et  techniques  de 
la  mesure  PID  pour  un  auditoire  scientifique.  La  mesure  actuelle  sert  de  point  de  depart  a  la  mise 
en  place  d’une  mesure  globale  des  activites  RSR  maritimes. 
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Executive  Summary 


A  Metric  for  Maritime  Intelligence,  Surveillance,  and 
Reconnaissance  (ISR)  -  Probability  of  Identification 

S.A.  Horn,  N.L.  Carson,  A.F.  Wind;  DRDC  CORA  TM  2009-037;  Defence  R&D  Canada  - 

CORA;  September  2009. 

Background:  There  is  an  operational  requirement  for  the  Canadian  Forces  (CF)  to  maintain  mar¬ 
itime  Intelligence  Surveillance  and  Reconnaissance  (ISR)  capability  in  support  of  Canadian  national 
defence  and  Canadian  national  security.  The  information  collected  from  maritime  ISR  activities  is 
compiled  into  the  Recognized  Maritime  Picture  (RMP)  through  Command  Control  Communica¬ 
tions  Computers  (C4)  activities.  Strategically,  the  Canadian  Navy  has  been  modernizing  by  combin¬ 
ing  information  from  new  sensor  technologies  and  Other  Government  Department  (OGD)  sources, 
with  data  being  shared  through  Marine  Security  Operations  Centre  (MSOC)  activities.  Improving 
the  quantity  and  quality  of  data  from  activities  such  as  this  also  results  in  an  improved  RMP,  and 
the  benefits  of  these  new  activities  can  and  should  be  measured. 

The  information  being  reported  to  the  operational  commander  must  be  useful.  Probability  of 
Identification  (P/d)  is  a  fundamental  metric  that  not  only  indicates  the  effect  of  surveillance,  but 
it  is  also  represents  identification:  the  first  step  to  a  t  lire  at  assessment.  Visual  identification  was 
requested  by  the  operational  commander  as  the  most  trusted  form  of  surveillance  information,  even 
though  other  sensors  can  provide  the  identity  of  targets.  P/d  starts  to  give  the  operational  comman¬ 
der  a  true  sense  of  the  risk  associated  with  gaps  in  maritime  surveillance. 

Principal  results:  P/d  is  calculated  based  on  the  frequency  of  patrols  ( T regular),  the  transit  times  of 
maritime  targets  (P),  the  effectiveness  of  a  surveillance  patrol  (77 ),  and  noting  that  multiple  detec¬ 
tions  and  identifications  do  not  add  any  additional  information  about  the  identity  of  the  interrogated 
target.  Considering  a  single  target  in  one  area  yields: 


P/D  = 


T 

T/77 


1-(1 


T  <  T 
r\)T/T  T>  t 


The  definition  is  then  expanded  to  incorporate  multiple  patrol  areas,  and  multiple  traffic  routes. 
PiD(Route Average)  is  a  cumulative  result,  which  simplifies  interpretation  of  the  effect  of  multiple  patrols 
and  ISR  activities  over  time. 


■ / D  (Route  Average) 


E^l  PlD{Route)i 

Nr 


Where  Pm(Route),  is  the  P/d  of  all  traffic  on  route  i  over  all  transited  patrol  zones  and  there  arc  Nr 
routes. 
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Significance  of  results:  Because  averaged  route  Pm  directly  measures  the  effect  of  surveillance 
activities,  the  identification  of  vulnerabilities  is  reduced  to  simply  observing  a  Pm  map  of  the  Area 
of  Responsibility  (AOR).  It  is  a  number  that  can  be  easily  compared  over  different  times  of  year, 
and  different  areas.  Pm  is  simply  a  number  for  each  zone  indicating  the  percent  of  incoming  vessels 
within  that  zone  that  are  expected  to  be  identified,  it  is  not  dependent  on  how  many  vessels  are 
transiting  within  the  zone,  and  there  is  no  need  to  understand  the  specific  traffic  patterns  within  the 
AOR  to  properly  interpret  its  meaning.  The  metric  presents  a  clear,  easy  to  interpret,  quantitative 
measure  of  the  maritime  ISR  activities  over  a  period  of  time. 

Future  work:  With  constant  addition  of  new  sensors  and  fusion  tools,  it  will  be  important  to  evolve 
the  ISR  effect  metric  to  account  for  these  new  capabilities.  Although  the  Pm  metric  presented  here 
is  calculated  for  visual  identification,  an  extension  of  the  metric  that  incorporates  an  information 
trust  coefficient  will  enable  Pm  reporting  of  non-visual  ISR  activities.  Work  towards  defining  em¬ 
pirical  coefficients  through  data  collection  is  being  pursued  by  the  Joint  Task  Force  (Pacific)  (JTFP) 
Operational  Research  Team  (ORT). 

The  next  step  in  advancing  the  utility  of  Pm  is  to  create  a  definition  that  will  enable  approximate 
real  time  and  high  resolution  Pm  reporting  including  the  extrapolation  of  Pm  into  the  near  future. 
This  capability  is  being  developed  by  the  Joint  Task  Force  (Atlantic)  (JTFA)  ORT. 

This  paper  addresses  the  identification  component  of  surveillance.  Measuring  the  tracking  compo¬ 
nent  of  surveillance  is  also  important,  and  the  definition  of  a  quality  of  tracking  metric  which  can 
be  reported  in  parallel  to  Pm  will  significantly  add  to  the  ability  to  measure  surveillance  effect.  A 
quality  of  tracking  metric  has  yet  to  be  formally  defined.  Work  to  define  a  complete  set  of  metrics 
is  being  done  jointly  by  JTFA  and  JTFP  OR  teams. 
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Contexte  :  Les  Forces  canadiennes  (FC)  ont  besoin  d’une  capacite  de  renseignement,  de  sur¬ 
veillance  et  de  reconnaissance  (RSR)  maritime  a  l'appui  de  la  defense  et  de  la  securite  du  Ca¬ 
nada.  Les  renseignements  recueillis  a  parti r  des  activites  RSR  maritimes  sont  compiles  dans  le 
tableau  de  la  situation  maritime  (TSM)  par  le  biais  d’ activites  de  commandement,  de  controle, 
de  communications  et  d’informatique.  Sur  le  plan  strategique,  la  marine  canadienne  s’est  moder- 
nisee  en  combinant  les  renseignements  obtenus  des  nouvelles  technologies  des  capteurs  et  d’autres 
ministres,  les  donnees  etant  partagees  par  le  biais  d’ activites  du  Centre  des  operations  de  securite 
maritime  (COSM).  L' amelioration  de  la  quantite  et  de  la  qualite  des  donnees  tirees  de  telles  activites 
debouche  egalement  sur  F amelioration  du  TSM.  Les  avantages  de  ces  nouvelles  activites  peuvent 
et  doivent  etre  mesures. 

Les  renseignements  signales  au  commandant  operationnel  doivent  etre  utiles.  La  probabilite  d'iden- 
tification  (Pm)  est  une  mesure  fondamentale  qui,  non  seulement  montre  les  effets  de  la  surveillance, 
mais  represente  aussi  F identification,  la  premiere  etape  de  l’evaluation  d’une  menace.  Le  comman¬ 
dant  operationnel  estime  que  F  identification  visuelle  est  la  forme  de  renseignement  de  surveillance 
la  plus  digne  de  confiance,  meme  si  des  capteurs  peuvent  fournir  Fidentite  des  cibles.  La  Pm  com¬ 
mence  a  fournir  au  commandant  operationnel  un  sens  reel  du  risque  associe  aux  lacunes  de  la 
surveillance  maritime. 

Resultats  principaux  :  La  Pm  est  calculee  a  partir  de  la  frequence  des  patrouilles  (t regular),  du 
temps  de  transit  des  cibles  maritimes  (T),  de  l’efficacite  d’une  patrouille  de  surveillance  (77 )  et  en  no- 
tant  que  des  detections  et  des  identifications  multiples  n’ajoutent  aucun  renseignement  supplementaire 
sur  Fidentite  de  la  cible  interrogee.  En  tenant  compte  d’une  cible  unique,  dans  un  seul  domaine,  on 
obtient  : 


Pid  = 


T 

■c/n 


1-(1 


T  <  T 
r,)TN  T  >  t 


La  definition  est  alors  elargie  de  maniere  a  integrer  des  zones  de  patrouille  et  des  routes  de  na¬ 
vigation  multiples.  Le  Pm(Route Average)  est  un  tcsultat  cumulatif  qui,  avec  le  temps,  simplifie  l'in- 
terpretation  de  l’effet  des  patrouilles  multiples  et  des  activites  RSR. 


-ID(Route  Average) 


E^l  PlD(Route)i 

Nr 
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PiD(Route),  est  la  Pid  de  tout  le  trafic  sur  l’itineraire  i  (toutes  les  zones  de  patrouille  etant  traversees), 
et  il  y  a  au  total  des  routes  Nr. 

Pertinence  des  resultats  :  Etant  donne  que  la  (Pin)  de  la  route  moyenne  mesure  directement  les 
effets  des  activites  de  surveillance,  1' identification  des  vulnerabilites  est  reduite  a  la  simple  observa¬ 
tion  d’une  carte  PID  de  la  zone  de  responsabilite  (ZR).  II  s’agit  d'un  nombre  pouvant  etre  facilement 
compare  a  differentes  periodes  au  cours  d’une  annee  et  a  differents  secteurs.  La  P/p  est  un  nombre 
attribue  a  chaque  zone  indiquant  le  pourcentage  de  navires  entrant  dans  cette  zone  qui  doivent  etre 
identifies ;  elle  n’est  pas  reliee  au  nombre  de  navires  qui  transitent  dans  la  zone  et  il  n’est  pas 
necessaire  de  comprendre  la  circulation  particuliere  de  la  ZR  pour  bien  en  comprendre  la  signifi¬ 
cation.  La  methode  presente  une  mesure  facile  a  interpreter,  claire  et  quantitative  de  l’efficacite  des 
activites  RSR  maritimes  pendant  une  periode  definie. 

Recherche  future  :  Avec  l’ajout  constant  de  nouveaux  capteurs  et  d’outils  de  fusion,  il  sera  impor¬ 
tant  de  developper  la  mesure  des  effets  RSR  afin  de  prendre  en  compte  les  nouvelles  capacites.  Bien 
que  la  mesure  de  la  Pip  presentee  ici  soit  en  fonction  d’une  identification  visuelle,  un  elargissement 
de  la  mesure  integrant  un  coefficient  de  renseignement  permettra  le  signalement  P/p  d’ activites  RSR 
non  visuelles.  L’equipe  de  recherche  operationnelle  (ERO)  de  la  Lorce  operationnelle  interarmees 
du  Pacifique  (LOIP)  continue  de  travailler  a  definir  des  coefficients  empiriques  par  le  biais  de  la 
collecte  de  donnees. 

La  prochaine  etape  pour  faire  avancer  l’utilite  de  la  Pjp  consiste  a  formuler  une  definition  qui  per¬ 
mettra  temps  reel  le  signalement  a  haute  resolution,  et  au  temps  reel,  de  la  P/p.  incluant  l’extrapo- 
lation  de  la  P/p  dans  un  avenir  rapproche.  L’equipe  de  recherche  operationnelle  (ERO)  de  la  Lorce 
operationnelle  interarmees  du  Pacifique  (LOIP)  travaille  a  l’elaboration  de  cette  capacite.  Le  present 
document  porte  sur  l’element  identification  de  la  surveillance. 

La  mesure  du  suivi  de  la  surveillance  est  egalement  importante.  La  definition  d’une  mesure  de  suivi 
pouvant  etre  parallele  a  la  P/p  ajoutera,  de  maniere  significative,  a  la  capacite  de  mesurer  les  effets 
de  la  surveillance.  Une  qualite  de  la  mesure  du  suivi  reste  a  formuler.  Le  travail  de  definition  d’un 
ensemble  complet  de  mesures  est  execute  conjointement  par  les  equipes  de  recherche  operationnelle 
de  la  LOIA  et  de  la  LOIP. 


VI 
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1  Introduction 


The  work  presented  in  this  paper  stems  from  Defence  Research  and  Development  Canada  (DRDC) 
Centre  for  Operational  Research  and  Analysis  (CORA)  support  to  the  Canadian  Forces  (CF).  There 
is  an  operational  requirement  for  the  CF  to  maintain  Intelligence  Surveillance  and  Reconnaissance 
(ISR)  capability  in  support  of  Canadian  national  security  and  Canadian  national  defence.  ISR  is 
a  core  role  of  the  Canadian  Navy,  and  the  importance  of  improving  Canada’s  ISR  capability  is 
outlined  in  the  Canadian  Navy’s  ISR  Blueprint  to  2010  [1],  Although  maritime  ISR  enhancement 
has  recently  come  to  the  forefront  of  the  Navy’s  priorities,  supporting  maritime  ISR  enhancement 
activities  has  been  an  ongoing  role  of  DRDC  CORA  over  the  past  decade.  Unfortunately,  there 
is  little  open  source  academic  literature  on  measuring  the  ability,  effects  and  performance  of  ISR 
activities.  As  such,  this  work  integrates  and  builds  upon  multiple  pieces  of  previous  work,  several 
of  which  will  be  outlined  and  referred  to  within  this  report.  The  main  sources  for  this  research  were 
DND  internal  reports. 

This  paper  is  one  of  three  unclassified  papers  on  this  topic  that  are  directed  to  different  audiences. 
Here,  the  theory  and  method  behind  the  Probability  of  Identification  (Pin)  metric  is  described  for 
the  scientific  community.  This  metric  was  developed  in  support  of  Concept  of  Operations  (CONOP) 
LEVIATHAN  and  further  refined  by  Joint  Task  Force  (Atlantic)  (JTFA)  and  Joint  Task  Force 
(Pacific)  (JTFP)  OR  Teams.  A  second  paper  describes  the  OR  support  for  the  development  of 
JTFA  CONOP  LEVIATHAN  [2],  and  how  the  Pm  metric  is  used  to  report  on  ISR  activities  to  the 
commander  [3].  The  third  paper  documents  the  Surveillance  Analysis  Workbook  (SAW)  for  the 
military  operators  for  use  as  a  general  background  and  instruction  manual  for  the  generation  of  re¬ 
ports  [4].  The  SAW  paper  also  describes  the  method  used  to  implement  the  Pm  metric  describing 
how  data  is  input,  processed,  and  the  results  visualized.  A  fourth  classified  Technical  Note  docu¬ 
ments  the  method  that  was  used  to  derive  traffic  routes  for  the  JTFA  and  JTFP  implementations  of 
the  Pm  metric  in  SAW  [5]. 

1.1  Maritime  ISR 

Maritime  ISR  is  a  part  of  Command  Control  Communications  Computers  (C4)  ISR,  also  referred 
to  as  C4ISR,  and  is  accounted  for  under  an  umbrella  concept  called  Maritime  Domain  Awareness 
(MDA).  Maritime  ISR  is  a  feedback  system  in  which  the  deployment  of  surveillance  assets  is 
planned  or  cued,  the  assets  are  deployed,  and  the  collected  information  from  these  assets  is  compiled 
into  a  Recognized  Maritime  Picture  (RMP).  In  addition  to  providing  current  situational  aw  areness, 
the  RMP  guides  command  and  intelligence  activities  for  future  ISR  planning  (in  both  the  near  and 
far  term).  Within  this  loop,  the  quality  of  the  RMP  is  key  to  the  CF  being  able  to  achieve  its  security 
goals.  Therefore,  improving  the  efficiency  of  maritime  ISR  is  one  path  to  improving  the  RMP. 

The  RMP  is  compiled  from  ISR  activities  through  C4.  Strategically,  the  Canadian  Navy  has  been 
modernizing  its  C4  capabilities  by  combining  information  from  new  sensor  technologies  and  Other 
Government  Department  (OGD)  sources,  and  sharing  data  through  Marine  Security  Operations 
Centre  (MSOC)  activities1.  Improving  the  quantity  and  quality  of  data  from  activities  such  as 

’The  MSOC  consists  of  Department  of  National  Defence  (DND),  Transport  Canada  (TC),  Royal  Canadian  Mounted 


DRDC  CORA  TM  2009-037 


1 


these  also  results  in  an  improved  RMP,  and  the  benefits  of  these  new  activities  should  and  can  be 
measured. 

Maritime  ISR  activities  conducted  by  JTFA  and  JTFP  arc  a  fundamental  component  for  the  mitiga¬ 
tion  of  risk  from  maritime  threats  to  Canada.  Maritime  surveillance  is  conducted  by  a  combination 
of  assets  including  Long  Range  Patrol  Aircraft  (LRPA)  missions,  contracting  commercial  flights 
(such  as  Provincial  Aerospace  Ltd.  (PAL)),  surface  patrol  vessels,  collecting  unclassified  position 
reports,  and  using  other  national  and  international  assets.  OGDs,  such  as  DFO  and  TC,  also  conduct 
maritime  surveillance  activities,  in  support  of  their  respective  mandates. 

1.2  History 

Maritime  ISR  research  has  been  an  active  area  of  research  within  DRDC  CORA  for  many  years.  The 
requirement  to  improve  ISR  capability  was  highlighted  in  the  late  1990’s,  when  several  vessels  were 
discovered  transporting  illegal  immigrants  from  Asia  to  the  west  coast  of  Canada.  In  response,  the 
JTFP  Operational  Research  Team  (ORT)  supported  the  Canadian  Navy  in  the  creation  of  Operations 
Plan  (OPLAN)  “SEA  LION”  [6].  The  JTFP  OP  SEA  LION  work  by  Gauthier  et  al.  resulted  in  the 
most  recent  ISR  reporting  templates  to  the  JTFP  operational  commander  [7], 

SEA  LION  reported  the  performance  of  ISR  activities  by  defining  a  minimum  and  desired  standard 
for  the  frequency  of  maritime  patrols.  These  revisit  requirements  were  defined  by  asserting  a  fun¬ 
damental  level  of  ship  detection  within  the  Area  of  Responsibility  (AOR),  and  from  that  calculating 
the  corresponding  level  of  surveillance  required  to  achieve  that  level  of  detection. 

Following  up  on  the  success  of  OP  SEA  LION,  the  JTFA  ORT  began  work  on  a  similar  plan  for  the 
East  coast  with  JTFA  CONOP  LEVIATHAN  [2],  The  work  highlighted  avenues  for  improvement 
to  this  type  of  ISR  activity  and  reporting.  Improvements  included  a  re-vamping  of  the  reporting 
templates,  and  redefining  the  standard  metric  from  revisit  to  Pm.  Calculating  and  reporting  Pm  is 
a  significant  improvement  to  ISR  reporting,  however  there  still  remains  significant  room  for  further 
improvements. 

Shortly  after  the  implementation  of  the  Pm  metric,  the  JTFA  and  JTFP  operational  commanders 
requested  that  the  ISR  reporting  templates  be  consolidated.  During  this  consolidation,  the  SAW 
tool  was  developed  to  enable  the  Regional  Joint  Operations  Centers  (RJOCs)  to  generate  reports 
with  a  common  look  and  feel.  The  implementation  of  SAW  is  discussed  in  another  paper  [4]. 
Because  of  the  simplicity  of  the  implementation,  and  the  success  of  the  metric.  Pm  is  now  used  by 
both  JTFP  and  JTFA  to  report  on  ISR  activities. 

1.3  Why  Probability  of  Identification 

The  information  being  reported  to  the  operational  commander  must  be  useful.  Visual  identification 
was  requested  by  the  JTFA  operational  commander  as  the  most  trusted  form  of  surveillance  infor¬ 
mation,  even  though  other  sensors  can  also  help  identify  targets.  Identification  is  the  first  step  to  a 

Police  (RCMP),  Canadian  Border  Security  Agency  (CBSA)  and  Department  of  Fisheries  and  Oceans  (DFO),  which  is 
represented  by  both  the  Conservation  and  Protection  (C&P)  and  Canadian  Coast  Guard  (CCG)  divisions. 
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threat  assessment,  and  therefore.  Pm  also  provides  an  indication  of  the  ability  to  make  threat  assess¬ 
ments  from  the  RMP.  For  this  reason,  the  Pu>  metric  starts  to  give  the  operational  commander  a  true 
sense  of  the  risk  associated  with  gaps  in  maritime  surveillance.  The  Pm  metric  indicates  the  risk  of 
an  incoming  vessel  not  being  identified,  and  therefore  the  risk  of  the  vessel  not  being  assessed  as  a 
threat. 

While  identification  alone  provides  the  information  vital  to  making  a  threat  assessment,  tracking 
is  also  required  to  respond  to  any  threats.  Both  known  and  unknown  vessels  are  tracked  in  the 
RMP,  and  identification  is  not  always  required  for  tracking.  A  good  example  of  this  is  the  Multi¬ 
sensor  Integration  within  a  Common  Operating  Environment  (MUSIC)  Technology  Demonstration 
Program  (TDP)  in  tracking,  as  the  MUSIC  algorithms  track  known  unknown  vessels  in  addition  to 
known  vessels  [8].  Ideally,  given  sufficient  update  rates,  vessels  that  are  identified  only  once  can 
be  tracked  in  the  RMP  without  the  need  for  additional  visual  identification.  Although  tracking  is 
important  for  acting  on  a  threat  assessment,  the  tracking  component  of  the  surveillance  analysis  is 
not  discussed  in  this  paper. 

The  metrics  for  identification  and  tracking  have  been  effectively  de-convolved  in  the  following 
discussions.  However,  quality  of  tracking  is  not  ignored;  instead,  it  is  to  be  dealt  with  in  future 
work.  Separate  metrics  for  tracking  and  identification  effectiveness  simplifies  the  problem,  allowing 
a  divide  and  conquer  approach.  The  metric  with  the  highest  impact  for  reporting  ISR  effect,  the  Pm 
metric,  was  chosen  as  a  first  approach.  By  using  only  a  Pm  metric,  it  is  assumed  that  targets  are 
tracked,  but  the  effectiveness  of  the  tracking  is  not  considered  when  calculating  and  reporting  Pm- 

1.4  Aim  and  Intent 

The  aim  of  this  paper  is  to  outline  a  method  for  measuring  and  scoring  the  quality  of  ISR  activities  in 
an  operational  context.  The  Pm  metric  was  identified  as  an  effective  measure  of  the  desired  effects 
of  ISR,  and  has  been  adopted  by  Canada’s  regional  operational  commands  as  a  standard  method 
of  ISR  reporting.  The  derivation  of  the  Pm  metric  and  the  method  of  calculation  is  presented  and 
discussed  in  the  following  chapters. 

While  this  paper  is  intended  for  the  scientific  community,  the  theoretical  basis  presented  here  is 
intended  to  serve  as  a  reference  for  other  papers.  An  operational  discussion  intended  for  a  military 
audience  describing  the  use  of  Pm  for  operations  is  described  in  a  CONOP  LEVIATHAN  paper 
[2,3].  The  implementation  of  the  metric  as  a  computer  program  is  described  in  the  SAW  paper 
which  is  tailored  for  operators  [4],  The  marine  traffic  models  used  in  SAW  are  available  in  a 
classified  Technical  Note  [5]. 
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2  Methods 


This  chapter  walks  through  the  process  of  deriving  the  Pm  metric.  It  begins  with  a  general  descrip¬ 
tion  of  maritime  ISR  metrics,  and  then  introduces  the  concepts  of  patrol  effectiveness  and  then  Pm 
for  random  and  regular  patrols.  The  Pm  metric  is  then  developed  further  through  its  calculation 
over  time,  through  multiple  areas,  and  across  multiple  routes. 

2.1  Metrics 

Metrics  for  ISR  have  been  extensively  studied  and  discussed  in  previous  papers  [9-13].  Some  of 
the  detailed  discussions  in  those  papers  arc  classified,  but  the  portions  covered  here  arc  not.  An 
organizational  layout  of  the  metrics  has  already  been  established  [11]. 

Maritime  ISR  metrics  can  be  divided  into  two  general  levels:  low-level  metrics,  which  assess  the 
capabilities  inherent  to  ISR  systems,  and  high-level  metrics,  which  assess  the  achievements  and  out¬ 
comes  of  ISR  objectives.  A  low-level  metric  is  generally  called  a  Measure  of  Performance  (MOP), 
and  a  high-level  metric  is  called  a  Measure  of  Effectiveness  (MOE).  It  has  been  noted  elsewhere 
that  “of  prime  importance  to  measuring  effectiveness  is  a  clear  statement  of  objectives”  [10].  This 
quote  serves  to  both  define  an  MOE  and  identifies  the  reason  why  they  are  difficult  to  define.  One 
does  not  always  have  the  luxury  of  a  precise  statement  of  objectives,  as  the  commander’s  guidance 
is  usually  very  general. 

Because  measuring  effect  is  generally  a  difficult  task,  using  specific  MOPs  for  measuring  effect  has 
been  proposed  [10].  For  example,  to  acquire  a  general  picture  of  vessel  activity,  one  can  look  at 
the  number  of  incoming  reports  and  the  coverage  of  those  reports.  Unfortunately,  while  this  may 
provide  some  indication  of  effectiveness,  it  fails  to  address  how  operational  objectives  arc  being 
met. 

Revisit  is  a  measure  of  presence  and  employment  of  assets  and  is  considered  an  MOP;  it  will  not  be 
discussed  in  detail  in  this  paper.  The  Pm  metric  measures  identification,  and  identification  supports 
the  Navy’s  ISR  objective  of  identifying  potential  threats,  thus  reporting  the  effect  of  surveillance 
activity.  Therefore,  Pm  is  considered  a  MOE. 

Since  April  2008,  the  reporting  and  assessment  of  maritime  ISR  activities,  via  OP  SEA  LION  and 
CONOP  LEVIATHAN,  is  accomplished  using  two  metrics.  They  are:  1)  the  previously  used  revisit 
metric  and  2)  the  newer  probability  of  (visual)  ID  metric,  or  Pm. 

Reporting  of  maritime  ISR  activities  using  both  effect  and  performance  measures  provides  informa¬ 
tion  to  the  command  on  how  efficiently  assets  were  utilized,  and  the  output  of  their  use.  While  both 
aspects  of  maritime  ISR  reporting  are  important,  this  paper  focuses  on  the  effects  based  metric.  Pm- 
The  following  sections  will  walk  through  the  derivation  of  the  Pm  equations. 

2.2  Patrol  effectiveness 

The  effectiveness  of  a  maritime  surveillance  patrol  is  represented  by  the  coefficient  of  effectiveness, 
rj,  which  represents  the  probability  of  detecting/identifying  a  vessel  that  is  present  in  the  zone 
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during  a  surveillance  flight.  As  a  pre-cursor  to  calculating  the  P/d  of  a  vessel,  the  effectiveness  of 
patrols  is  considered.  There  arc  three  dependencies  to  the  effectiveness  of  a  patrol:  the  fraction  of 
an  area  covered  by  the  patrol  (Coverage),  the  probability  of  detecting  a  vessel  if  the  patroller  is  in 
range  (Pd),  and  the  probability  of  identifying  a  detected  vessel  (P/d\d)-  The  effectiveness  coefficient, 
described  in  detail  below,  is  used  later  when  calculating  the  probability  of  identifying  a  vessel. 


V  =  Coverage  ■  PD  •  Pm\n  V  G  [0, 1]  (1) 

2.2.1  Probability  of  Detection 

In  Equation  1,  Probability  of  Detection  (Pd)  is  a  detection  coefficient  generally  based  on  anecdotal2 
assessments  about  LRPA  flights.  The  probability  of  identification  of  a  vessel  while  it  is  in  the 
surveillance  area  is  dependent  on  the  ability  to  detect  the  vessel  (which  may  be  done  with  non- 
LRPA  resources)  and  then  subsequently  identify  the  vessel,  all  while  the  patroller  and  the  vessel  arc 
still  in  the  surveillance  area.  P/d  jo  is  based  on  the  number  of  detected  vessels  that  arc  identified.  For 
the  purposes  of  this  report,  the  assumptions  arc  that  all  detections  arc  identifications  (P/d\d  =1)  and 
they  arc  done  by  the  same  platform  (i.e.  a  surveillance  aircraft).  For  vessels  within  a  patrol  zone, 
the  combination  of  Pd  and  P/d\d  gives  the  probability  that  a  patrol  will  identify  vessels  when  given 
the  opportunity. 

Partial  identification  is  the  detection  of  contacts  for  which  we  know  where  they  may  be,  but  not  who 
they  arc  (i.e.  no  name)  where  detection  only  refers  to  knowledge  of  where  a  contact  is.  The  detailed 
discussion  on  the  criteria  for  partial  identification,  given  detection,  is  beyond  the  scope  of  this 
paper.  Partial  identifications  will  not  provide  improvements  to  the  P/d  metric  used  here  because,  as 
stated  earlier,  tracking  is  de-convolved  from  identification.  However,  parti al  identification  provides 
a  benefit  to  the  RMP  when  tracking  vessels  and  fusing  multiple  data  sources.  The  contribution 
of  partial  identification  to  surveillance  would  be  captured  in  a  quality  of  tracking  metric,  to  be 
addressed  in  subsequent  papers. 

2.2.2  Coverage 

The  “coverage”  of  the  patrol  is  the  percentage  of  the  surveillance  area  that  is  covered  for  which  the 
probability  of  detection  coefficient  is  valid  (i.e.  the  patroller  is  actively  observing).  This  is  usually 
accomplished  at  a  simplified  level  by  combining  a  cookie-cutter  model  of  radar  coverage  and  the 
surveillance  flight  path.  Consequently,  the  value  obtained  for  patrol  effectiveness  calculated  this 
way  should  be  viewed  as  an  estimate  and  not  a  precise,  scientifically  derived  value. 

Typically,  surveillance  planning  is  done  based  on  predefined  surveillance  areas,  with  no  overlap 
between  areas;  i.e.  a  gridded  surveillance  area.  Within  these  areas  the  crew  attempts  to  detect 
and  identify  all  vessels.  The  fraction  of  the  assigned  areas  covered  during  the  patrol  is  called  the 
coverage  scaling  factor.  If  a  portion  of  an  area  is  not  searched,  then  any  vessels  in  the  missed  area 

2The  Pd  value  is  determined  through  observations  by  surveillance  experts.  This  is  done  by  a  comparison  of  visual 
and  radar  observations  during  LRPA  flights.  Pq  is  estimated  by  the  percent  of  expected  vessels  that  were  detected  and 
identified. 
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can  not  be  detected  and  identified.  For  consistent  ISR  planing  and  reporting,  the  AORs  for  each 
coast  have  been  divided  into  standard  patrol  areas,  or  zones.  A  surveillance  flight  may  be  tasked  to 
search  an  area  that  may  not  be  a  complete  zone  but  rather  a  partial  area  of  one  or  more  zones.  In 
this  case,  for  reporting  purposes,  the  coverage  factor  for  each  partially  or  completely  covered  zone 
is  determined.  During  maritime  surveillance  patrols,  it  is  important  to  keep  track  of  the  amount  of 
the  assigned  surveillance  area  that  is  covered.  A  single  surveillance  patrol  is  considered  successful 
when  100%  coverage  is  achieved  for  an  assigned  area.  When  a  surveillance  patrol  covers  less  than 
the  assigned  area,  the  amount  of  area  not  covered  will  reduce  the  patrol  effectiveness  in  a  linear 
manner. 

If  the  patrol  asset  always  covered  the  entire  assigned  zones,  then  the  coverage  factor  would  become 
binary  and  one  would  simply  need  to  know  which  zones  were  covered.  This  would  continue  to  be 
the  case  even  if  the  zones  were  replaced  by  either  very  small  areas  or  continuous  functions  rather 
than  discrete  areas.  To  model  the  covered  area  as  either  continuous  functions  or  finite  elements 
would  require  detailed  information  on  the  flight  path  and  swath  width  throughout  the  flight. 

The  gridded  surveillance  area  approach  is  currently  easier  for  planners  to  manage,  but  it  also  means 
that  measurement  of  patrol  effectiveness  is  limited  to  the  preset  surveillance  area.  It  is  conceivable 
that  areas  of  interest  could  also  encompass  portions  of  a  single  or  multiple  surveillance  areas,  so 
there  is  no  easy  way  to  reflect  patrol  effectiveness  based  on  coverage  other  than  for  a  single  full 
area. 

If  the  surveillance  area  is  not  gridded,  then  surveillance  areas  can  be  dynamically  assigned  based 
on  priorities  of  that  week,  with  a  fall  back  position  to  “pseudo-gridded”  surveillance  areas  if  current 
events  do  not  dictate  specific  actions.  Due  to  the  nature  of  a  dynamic  surveillance  area  (i.e.  an 
irregular  shaped  area),  it  is  important  to  capture  the  flight  path  and  surveillance  swath  of  each  patrol 
in  sufficient  detail  to  allow  the  flight  to  be  reconstructed  to  estimate  the  coverage  of  the  dynamic 
surveillance  area.  This  method  is  technically  feasible  but  very  challenging  to  implement,  so  it  is  not 
currently  done  as  a  standard  practice.  The  real  coverage  is  an  area  for  future  development  towards 
more  detailed  planning  and  reporting  that  would  not  depend  on  large  and  arbitrary  standard  zones. 

2.3  Probability  of  Intercept 

Let  us  now  consider  the  surveillance  of  a  single  area  over  a  period  of  time  with  a  transiting  vessel. 
For  any  vessel  within  the  area,  the  probability  of  identification  of  that  vessel  is  dependent  on  not 
only  the  ability  to  detect  the  vessel  (using  whatever  detection  means  possible),  but  also  the  chance 
that  a  patrol  will  be  in  the  area  before  the  vessel  completes  its  transit  through  the  area.  This  means 
that  the  frequency  of  patrols  and  the  time  it  takes  for  the  vessel  to  transit  through  the  area  arc 
important  factors  for  calculating  the  probability  of  identifying  a  vessel. 

On  the  ocean,  the  time  it  takes  for  a  vessel  to  transit  through  a  designated  surveillance  area  is 
usually  significantly  larger  than  the  time  it  takes  to  complete  a  surveillance  flight.  This  is  because  an 
aircraft  in  the  air  is  much  faster  than  the  standard  vessel  on  the  water  and  the  aircraft  will  complete 
its  flight  much  faster  than  the  vessel  can  transit  through  the  patrol  zone.  For  this  reason,  it  is  a 
reasonable  assumption  that  the  aircraft  patrols  are  essentially  instantaneous  compared  to  the  vessel 
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transit  time  [7].  It  follows  that  the  probability  of  intercepting  a  vessel  during  its  transit  of  the 
surveillance  area  is  therefore  essentially  the  same  as  the  probability  of  a  patrol  happening. 

The  time  between  patrols  is  of  high  interest  because  it  is  during  this  time  that  a  vessel  has  a  chance 
to  “sneak  by”  undetected.  Let  us  then  define  the  time  between  successive  patrols  to  be  the  revisit 
time,  which  is  constant  for  regular  patrols  and  not  constant  for  random  patrols.  Although  the  revisit 
times  are  not  constant  for  random  patrols,  the  average  revisit  time  over  a  significantly  long  period  of 
time  is  assumed  to  be  constant.  Pseudo-random  patrols  are  more  difficult  to  understand  than  purely 
regular  or  random  patrols.  Instead  of  directly  modeling  pseudo-random  patrols,  the  purely  random 
and  purely  regular  cases  are  modeled,  with  the  model  for  the  pseudo-random  patrols  case  falling 
somewhere  between  these  two  extremes. 

There  arc  many  factors  that  contribute  to  patrols  not  being  random:  patrols  happen  mostly  in  day¬ 
light,  they  arc  constrained  to  aircraft  servicing  schedules,  and  the  patrols  themselves  are  scheduled. 
An  example  of  the  pseudo-randomness  is  shown  in  Figure  1  using  a  time  line  of  patrol  events.  Label 
a  indicates  a  transiting  ship  (taking  a  time  T  to  transit)  which  was  intercepted  by  a  patrol  and  Label 
b  indicates  a  transiting  ship  that  was  not  intercepted. 


Time - ► 


Figure  1:  Distribution  of  fictional  patrol  events  on  a  line  plot  showing  a)  an  intercepted  transit,  and 
b)  a  missed  transit. 

2.3.1  Random  Patrol  Intercept 

For  random  patrols  over  time,  patrols  happen  independently  of  one  another  with  a  constant  Mean 
Revisit  Time  (MRT),  denoted  by  T,  between  patrols.  The  average  rate  of  patrols  is  then  X=  The 
MRT  is  the  average  amount  of  time  between  patrols  and  should  not  be  mistaken  with  the  revisit  rate 
(A),  which  is  the  average  number  of  patrols  in  a  given  time. 

Now  consider  the  following  scenario.  A  ship  enters  the  patrol  zone  where  random  patrols  happen 
with  a  MRT  of  t.  If  this  ship  was  gambling  on  not  being  detected,  the  longer  the  ship  stays  in  the 
patrol  zone,  the  more  likely  that  a  random  patrol  is  going  to  happen.  In  this  particular  case,  the 
patrol  has  an  instantaneous  revisit  time  of  t,-,  which  is  not  necessarily  equal  to  t.  To  “sneak  by” 
undetected,  the  ship  has  to  make  it  through  the  patrol  zone  before  the  next  patrol,  and  the  longer 
it  takes  to  do  it,  the  more  likely  the  ship  will  be  caught.  Also,  the  longer  it  takes,  the  probability 
that  there  has  not  yet  been  a  patrol  decays  exponentially,  and  the  ship’s  odds  of  success  decays 
exponentially.  After  a  time  t  in  the  patrol  zone,  the  probability  that  there  has  not  yet  been  a  patrol 
is  given  in  Equation  2. 


P(Zi  >t)=eT 


(2) 
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The  probability  of  a  random  patrol  happening  after  a  time  t  is  then  simply  1  minus  the  probability 
that  a  patrol  has  not  happened,  as  shown  in  Equation  3. 


P(Xi<t)  =  \-P(Xi>t)  =  \-el  (3) 

Making  the  assumption  that  patrols  arc  relatively  instantaneous,  the  probability  of  intercept  becomes 
the  same  as  the  probability  of  a  patrol  happening  during  the  transit  period.  If  the  transit  time  of  the 
ship  through  the  patrol  zone  is  T,  then  the  probability  of  intercept  is  simply  the  probability  of  a 
patrol  within  that  transit  time.  This  is  the  same  as  Equation  3  evaluated  at  t  =  T. 


^band  1  e 


Equation  4  is  the  probability  of  intercepting  a  ship  that  takes  a  time  of  T  to  transit  the  patrol  zone 
using  purely  random  patrols  with  a  MRT  of  x. 


2.3.2  Regular  Patrol  Intercept 

For  regular  patrols,  the  revisit  time  is  constant,  and  each  instantaneous  revisit  time  r,  is  equal  to 
x.  Now,  if  a  ship  enters  the  patrol  zone  which  is  patrolled  with  a  regular  revisit  time  of  x,  the  ship 
can  only  “sneak  by”  if  there  is  no  patrol  while  it  is  transiting  the  patrol  zone.  If  the  time  between 
patrols  (x)  is  less  then  the  time  that  it  takes  for  a  ship  to  transit  the  patrol  area  (T),  then  there  is  no 
chance  for  the  ship  to  transit  unnoticed.  The  probability  of  intercept  will  always  be  1  when  T  >  x. 
However,  if  the  transit  time  is  less  than  the  revisit  time,  then  there  is  an  opportunity  for  the  ship  to 
“sneak  by”  if  the  ship  times  its  transit  just  right. 

For  the  ship  to  make  it  through  the  patrol  zone  without  being  intercepted,  the  window  of  opportunity 
is  right  after  a  patrol.  However,  assuming  that  the  time  the  ship  enters  the  patrol  zone  is  random  (i.e. 
the  ship  does  not  know  the  patrol  schedule,  and  the  aircraft  does  not  know  the  ships  transit  schedule), 
then  the  probability  of  the  ship  entering  the  patrol  zone  outside  of  the  window  of  opportunity  is 
simply  the  ratio  of  the  transit  time  and  patrol  revisit  time.  The  probability  of  intercepting  is  simply 
the  fraction  of  time  that  the  window  of  opportunity  is  not  open.  Equation  5  gives  the  probability  of 
intercept  for  regular  patrols  with  a  revisit  time  of  x  and  a  ship’s  transit  time  of  T . 


p^={;  HI  (5) 

2.3.3  Alternate  Patrol  Intercept  Method 

There  are  a  few  models  to  calculate  the  probability  of  interception.  Martel  and  Nguyen  formulated 
the  probability  of  intercepting  intruders  using  a  combinatorial  method  [9].  They  assume  the  proba¬ 
bility  of  detecting  a  portion  of  transiting  vessels  was  a  binomial  distribution  with  a  mean  probability 
based  on  the  patrol  frequency  and  patrol  effectiveness.  Their  formulation  is  given  in  Equation  6,  in 
which  I  is  the  traffic  volume,  c  is  the  number  of  vessels  intercepted,  P(c  =  i)  is  the  probability  of 
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intercepting  i  vessels,  and  Pc  is  the  probability  of  detecting  a  vessel  for  a  given  patrol  frequency  and 
transit  time.  Because  their  formulation  depends  on  the  volume  of  traffic  in  the  patrol  area,  it  mod¬ 
els  both  the  increased  load  on  individual  patrollers  to  detect  vessels  and  the  effect  of  diminishing 
returns  from  additional  flying  hours.  However,  their  formulation  requires  a  priori  knowledge  of  the 
traffic  volume  to  calculate  the  probability  of  intercept,  even  though  traffic  is  what  the  flight  is  meant 
to  assess. 


p(C=i)  =  Q(pc)i(i-pcy-i  (6) 

The  binomial  nature  of  their  formulation  assumes  that  patrols  over  time  arc  randomly  binned.  In 
reality,  patrols  arc  only  pseudo-random. 

2.4  Probability  of  Identification 

The  next  step  in  the  analysis  is  to  make  the  connection  going  from  a  probability  of  intercept  to  a 
probability  of  identification.  In  Section  2.2,  the  patrol  effectiveness  coefficient  r]  was  introduced 
in  Equation  1  which  captured  the  various  components  required  to  go  from  intercepting  a  vessel  to 
identifying  a  vessel.  One  way  to  account  for  patrol  effectiveness  in  P/d  is  to  consider  the  P/  with  a 
scaled  revisit  time.  This  way,  the  probability  of  identification  is  the  probability  of  intercept  with  the 
revisit  time  divided  by  T] .  This  works  for  the  random  revisit  time  equation,  and  the  regular  revisit 
time  equation  for  T  <  t.  In  other  words,  the  P/d  contribution  from  a  patrol  with  a  low  effectiveness 
is  modeled  as  a  completely  effective  patrol  with  an  increased  revisit  time.  In  effect,  this  modifies 
the  contribution  of  each  revisit  towards  identification. 


Pid  =  E/(t')|t'=t/7] 


(V) 


For  random  patrols,  the  P/d  is: 


piDmnd  =  1-ePv  (8) 

For  regular  patrols  in  which  T  >  z,  this  method  does  not  work  because  we  have  already  assumed 
that  if  the  transit  time  is  larger  than  the  revisit  time,  then  the  probability  of  intercepting  should  be  1. 
Additionally,  for  frequent  patrols,  there  is  a  chance  that  the  vessel  will  be  intercepted  multiple  times. 
It  is  expected  that  the  probability  of  multiple  intercepts  increases  as  z  — ►  0,  and  the  model  for  the 
probability  of  identification  should  approach  100%.  The  P/d  when  T  >  z  is  modeled  by  combining 
the  patrol  effectiveness  from  multiple  intercepts.  With  this  model,  flying  more  frequently  increases 
the  probability  of  identification  when  there  are  multiple  opportunities  for  intercept.  Equation  9  gives 
the  formulation  used  for  the  regular  patrol  P/d- 


PlDreg 


T 

t/j] 


1-(1 


T  <z 
r\)T T  >  z 


(9) 


10 


DRDC  CORA  TM  2009-037 


where  1  —  r/  is  the  probability  of  not  detecting  a  ship  and  T/z  is  the  expected  number  of  revisits 
during  a  transit.  To  help  understand  what  these  functions  look  like,  it  is  useful  to  visualize  the  shape 
of  the  functions.  Figure  2  illustrates,  with  rj  =  1,  and  a  transit  time  T  =  6,  the  Pjd  for  random  and 
regular  patrols.  It  may  at  first  appear  counter-intuitive  that  random  patrols  provide  lower  Pjd  than 
regular  patrols,  but  this  makes  sense  due  to  the  uncertainty  in  revisit  times.  As  stated  before,  patrols 
are  neither  purely  regular  nor  random.  However,  patrols  are  arguably  more  regular  than  random, 
especially  when  analysing  the  Pjd  over  a  short  period  of  time.  Figure  2  is  a  good  illustration  of  the 
upper  and  lower  bounds  of  Pjd,  with  the  real  world  Pjd  from  pseudo-random  patrols  likely  closer 
to  the  upper  curve.  For  the  Pjd  metric,  the  regular  Pjd  formula  (Equation  9)  is  used  to  calculate  the 
probability  of  identification. 


Figure  2:  Pjd  with  rj  =  1  for  random  and  regular  patrols  (for  a  vessel  requiring  6  days  to  transit  an 
area  (t  =  6),  as  a  function  of  revisit  time,  z). 

In  this  formulation,  it  can  be  seen  that  the  maximum  value  of  Pjd  is  1 .  This  makes  sense  because 
successive  identifications  of  the  same  vessel  does  not  produce  any  new  information  about  its  iden¬ 
tity.  This  is  because  it  is  assumed  that  once  a  vessel  is  identified,  if  it  is  of  interest,  it  will  be  tracked 
using  non- visual  sensors  and  not  require  a  successive  mission  to  detect  and  identify  the  vessel  again. 
The  ability  to  continue  to  track  a  vessel  of  interest  is  important,  but  the  quality  of  that  tracking  is 
not  measured  through  the  Pjd  metric. 

It  should  be  noted  that  the  formula  for  Pjd  does  not  take  into  account  the  time  it  takes  to  patrol  the 
area,  as  it  was  assumed  earlier  that  the  patrols  could  be  taken  as  instantaneous.  The  accuracy  of 
Equation  9  is  reduced  when  the  time  it  takes  to  conduct  a  patrol  is  of  the  same  magnitude  as  the 
time  it  takes  to  transit  an  area.  An  example  of  when  this  may  happen  is  when  the  transit  time  for 
a  vessel  is  fast,  on  the  order  of  a  few  hours,  or  the  surveillance  of  the  area  takes  a  long  time  (e.g. 
surface  combatant). 
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2.4.1  Verification  of  Probability  Distributions 

To  verify  that  Equations  8  and  9  arc  valid  probability  distributions,  one  can  show  that  there  exists 
a  probability  density  function  for  each,  and  that  the  probability  density  functions  are  normalized 
under  the  condition  that  the  transit  time  T  >  0  and  revisit  time  z  >  0. 

For  Equation  8,  the  probability  density  function  is  the  derivative  with  respect  to  T . 

fraud  =  ~J^{Plmnd)  =  ~eT/V 

Equation  10  can  be  shown  to  be  normalized  over  all  z  >  0  by  integrating  for  all  possible  transit 
times. 

oo 

jfranddT  =  1  VT  >  0  (11) 

0 

For  Equation  9,  it  is  more  tricky  to  find  the  probability  density  function.  The  probability  distribution 
is  piece-wise  continuous,  and  can  therefore  be  written  as  a  single  expression  using  step-functions. 


Pioreg  =  • H(z-T )  +  (H(T)-H(z-T))  •  (l  -  (1  -  tj)  *)  (12) 

The  probability  density  function  for  Equation  9  is  now  derivative  of  Equation  12  with  respect  to  T. 

fr<!g  =  ~j7^{PlDreg)  (13) 

Equation  13  can  be  shown  to  be  normalized  for  all  z  >  0  by  integrating  for  all  possible  transit  times. 


J  fregdT  =  IVt  >  0  (14) 

o 

The  probability  density  functions  for  both  models  are  normalized  for  all  physically  valid  parameters 
for  z  and  T.  Therefore,  both  Equations  8  and  9  are  valid  probability  distributions. 

2.5  Probability  of  Identification  Within  a  Patrol  Area 

Using  Equation  9  (Pm  for  regular  patrols),  it  is  straightforward  to  calculate  the  probability  of  iden¬ 
tification  for  a  single  patrol  area.  Figure  3  shows  a  fictitious  surveillance  area  and  two  traffic  routes 
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Figure  3:  Example  map  showing  three  surveillance  regions  (A,B.  and  C)  and  two  threat  routes  (1 
and  2) 


which  will  be  used  to  illustrate  the  method  for  calculating  Pid ■  The  methodology  for  defining  the 
traffic  routes  for  a  real-world  AOI  is  described  in  Section  2.8. 

To  start,  the  Pid  in  region  A  along  route  1  will  be  calculated  first  without  considering  the  surveillance 
activities  in  B  and  C.  Afterwards,  the  metric  will  be  expanded  to  capture  the  contribution  from 
multiple  regions  and  then  multiple  routes.  Using  Figure  3,  if  the  period  for  surveillance  patrols  for 
region  A  is  48  hours,  the  average  time  for  a  vessel  to  transit  region  A  along  route  1  is  12  hours,  and 
the  patrol  effectiveness  is  50%,  then  the  Pid  is: 


p  _  _ 

rID(Region\A )  —  4g^Q  5 

=  0.125 

While  this  works  for  calculating  Pid  for  a  single  surveillance  area  and  one  route,  it  does  not  account 
for  surveillance  in  adjacent  routes.  The  case  of  incoming  traffic  routes  transiting  multiple  zones  is 
considered  next. 

2.6  Probability  of  Identification  through  Multiple  Areas 

Up  to  this  point,  the  Pid  values  were  calculated  for  a  single  surveillance  area.  When  a  vessel  transits 
through  multiple  contiguous  patrol  zones,  the  Pid  calculation  can  be  modified  such  that  it  becomes 
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the  probability  of  detecting  the  transiting  vessel  within  any  one  of  the  zones  on  its  course.  This  is 
called  the  route  Pm  and  is  denoted  PjD(Route)  ■  The  benefit  of  PjD(Route)  is  that  it  provides  a  measure 
of  identification  for  all  incoming  vessels  that  arc  transiting  through  a  route,  accounting  for  those 
that  may  have  already  been  identified  previously.  One  must  take  care  when  reading  a  Pm(Route) 
value  because  the  route  Pm  reflects  identification  from  surveillance  activities  in  multiple  zones. 
Increasing  surveillance  in  adjacent  zones  will  increase  the  route  Pm. 

The  route  Pm  for  Na  areas  is  simply  one  minus  the  product  of  not  detecting  the  vessel  within  any  of 
the  patrol  zones  along  its  path  and  is  given  by  the  following  formula3 : 


PlD(Route)  =  1  -  (1  -PlDi)  X  (1  ~  PlD.)  X-  ••  X  (1  ~  PidNa) 

=  i-nfii(i  -pid) 


(15) 


Equation  15  can  be  interpreted  as  the  Pm  of  a  vessel  along  a  route.  From  Equation  15,  it  follows  that 
increasing  the  Pm  in  any  area  along  the  vessel’s  route  will  increase  the  Pm(Route)  in  ah  subsequent 
surveillance  areas  the  vessel  transits  en  route  to  its  destination.  From  the  offshore  domestic  security 
perspective,  most  of  the  threats  arc  incoming  (i.e.  transiting  into  Canadian  waters).  By  limiting  the 
routes  to  consider  only  incoming  vessels,  the  calculation  is  greatly  simplified. 

As  an  example,  consider  route  1  in  Figure  3  that  crosses  both  regions  A  and  B.  Each  of  these  regions 
will  have  a  Pm(Route:i)>  which  indicates  the  probability  of  detecting  incoming  vessels  along  route  1 
that  terminate  in  that  region.  The  transit  time  across  region  A  is  12  hours  and  for  region  B,  it  is  6 
hours.  The  period  for  surveillance  patrols  for  region  A  is  48  hours  while  for  region  B  it  is  12  hours. 
Both  flights  have  a  patrol  effectiveness  of  50%.  To  calculate  the  route  1  Pm  terminating  in  region 
A,  the  vessel  must  transit  first  through  region  B,  and  then  A.  The  Pm  values  from  regions  A  and  B 
are  therefore  used. 

The  route  1  Pm  is  calculated  as  follows  for  region  A: 


PlD(Route:A\) 


=  0.344 

It  is  easy  to  see  from  these  calculations  that  increasing  Pm  in  region  B  has  the  effect  of  increasing 
the  route  Pm  in  region  A.  This  relationship  occurs  because  the  Pm  metric  accounts  for  vessels 
that  have  already  been  identified  prior  to  entering  the  latest  surveillance  zone.  Specifically,  in  this 
example,  the  metric  accounts  for  increased  identification  of  incoming  vessels  within  region  A  if 
they  have  already  been  identified  in  region  B.  In  other  words,  identified  vessels  along  route  1  in 

3It  should  be  noted  that  the  average  time  for  a  vessel  to  transit  a  surveillance  area  is  dependent  on  the  route  that  it  is 
following  and  therefore  the  transit  times  have  to  be  calculated  for  each  route. 


1  —  ( 1  —  Pida  )  x  ( 1  —  PmB ) 

12 


1-(1- 


:)  X  (1  — 


48/0.5 '  v  12/0.5' 
1- (1-0.125)  x  (1-0.25) 
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region  A  includes  those  vessels  that  were  identified  within  region  A  and  those  that  were  identified 
within  region  B. 

Calculating  the  route  Pm  for  region  B  is  much  simpler.  Because  the  method  only  considers  incoming 
routes,  there  is  no  need  to  consider  the  Pm  from  region  A,  as  that  would  be  an  outgoing  route.  The 
only  identified  vessels  in  region  B  will  be  those  that  were  identified  directly  within  region  B.  The 
route  Pjd  in  region  B  therefore  only  considers  the  effect  of  surveillance  activity  within  its  own  zone: 


P ID(Roule:B])  ~  PlDB 

=  0.25 

The  Pm  for  route  2  is  calculated  in  an  identical  way  as  for  route  1  above,  but  using  region  C  vice 
B.  In  the  real  world  traffic  model,  there  may  be  many  routes  entering  and  transiting  through  a 
surveillance  zone.  The  next  step  is  to  account  for  all  ways  that  traffic  may  enter  a  surveillance  zone 
by  considering  the  effect  of  all  of  the  routes  that  transit  into  and  through  each  zone. 

2.7  Average  Probability  of  Identification  Considering  Multiple 
Routes  through  Multiple  Areas 

To  measure  the  average  Pm  of  all  routes  that  cross  or  terminate  in  a  surveillance  area,  the  Pm  will 
depend  not  only  on  the  surveillance  activities  within  an  area,  but  also  on  the  Pm  on  routes  before 
traffic  enters  the  surveillance  area.  The  average  of  all  route  P/ps  in  a  surveillance  zone  gives  an 
overall  measure  of  identification  of  vessels  in  the  zone.  Therefore,  the  averaged  route  Pm  is  the 
effect  from  all  surveillance  activities  in  the  Area  of  Interest  (AOI).  For  a  region  with  Nr  routes 
terminating  in  it,  the  PID(Route Average)  is: 


_  E,=  l  PlD(Route:i)  , 

*ID(Route  Average)  jy  OW 

While  one  is  interested  in  identifying  all  incoming  vessels,  one  is  not  concerned  where  along  the 
route  that  vessel  was  identified.  Therefore,  the  Pm(Route  Average)  is  actually  the  probability  of  the 
vessel  being  identified  before  it  exits  through  the  surveillance  zone,  or  region,  no  matter  which 
route  that  vessel  is  traveling  on. 

If  there  is  information  available  on  a  known  threat,  or  knowledge  of  the  probability  of  a  threat 
along  each  route,  then  a  future  improvement  to  calculating  the  Pm(Route Average)  might  be  to  add  a 
weighting  to  each  route  indicating  the  likelihood  of  the  threat  using  that  route.  An  implementation 
of  this  weighting  may  take  advantage  of  Bayesian  methods  to  allocate  threat  probabilities  along  the 
routes.  However,  the  information  required  to  assign  a  weighting  will  often  not  be  available.  The 
simplest  and  most  straight  forward  method  is  to  assume  that  a  threat  can  come  from  anywhere,  and 
to  calculate  Pm  along  the  major  traffic  routes  to  maximize  the  number  of  interrogated  vessels. 

Of  the  various  probabilities  and  metrics  introduced  in  this  chapter,  the  Pm(Route Average)  is  the  most 
important  value  to  the  operational  commander.  It  is  a  number  that  can  be  easily  compared  over 
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different  times  of  year,  and  different  areas.  PiD(Route Average)  reduces  the  Fid  concept  to  a  number  for 
each  zone  indicating  on  average  the  percent  of  incoming  vessels  within  that  zone  that  arc  identified, 
it  does  not  matter  how  many  vessels  arc  transiting  within  the  zone,  and  there  is  no  need  to  understand 
the  traffic  patterns  within  the  AOR  to  correctly  interpret  its  meaning. 

Again  referring  to  Figure  3,  the  transit  times  along  route  2  is  8  hours  through  region  A  and  12 
hours  through  region  C.  The  period  for  surveillance  patrols  for  region  C  is  24  hours,  but  has  an 
effectiveness  of  75%. 

The  route  2  Pm  for  region  A  is: 


PlD(Route:A2)  =  1  -  (1  ~  PlDA)  X  (1  ~  PlDc) 

~  1-(1_  48/0.5  ^  X  24/0.75  ^ 
=  1 -(1-0.083)  x  (1-0.375) 

=  0.427 


And  then  the  averaged  route  Pjd  for  region  A  from  Equation  16  is: 


p  L/=l  PlD(Route)i 

‘ID  (Route Average:A)  2 

=  1(0.344  +  0.427) 

=  0.386 

Therefore,  the  average  Pm  for  identifying  threats  in  surveillance  zone  A  is  roughly  39%  along 
routes  1  and  2.  This  fictitious  example  served  to  illustrate  how  the  Pm  metric  is  calculated  for  a 
given  surveillance  grid  and  traffic  routes.  The  next  section  describes  the  methodology  for  defining 
real  world  traffic  routes. 

2.8  Identifying  routes  for  PID 

In  a  marine  environment,  there  are  nearly  an  infinite  number  of  routes  vessels  can  follow.  However, 
to  make  a  first  order  approximation  of  the  majority  of  traffic,  a  logical  approach  is  to  define  a  few 
major  traffic  routes  for  an  AOI.  Although  not  all  traffic  will  follow  these  routes  directly,  they  do 
capture  the  majority  of  transiting  vessels  and  the  typical  time  required  to  transit  through  an  area. 
Once  several  routes  are  defined,  the  overall  flow  of  traffic  in  the  AOI  is  captured,  and  the  distribution 
of  the  traffic  through  the  patrol  zones  can  be  approximated  by  the  identified  routes. 

To  save  fuel,  most  traffic  crosses  the  ocean  via  great  circle  routes.  Although,  inexperienced  naviga¬ 
tors  can  use  rhumb  line  routes,  commercial  shipping  transits  along  great  circle  routes  to  reduce  time 
and  distance  between  ports.  Additionally,  because  of  natural  choke -points,  many  routes  are  often 


16 


DRDC  CORA  TM  2009-037 


funneled  together  to  reduce  the  number  of  routes  to  major  ports  (e.g.  Unimak  Pass  4  or  the  Strait  of 
Belle  Isle  5).  For  the  case  of  Unimak  Island,  the  routes  move  south  in  the  winter  months  to  avoid 
severe  winter  conditions  along  the  Aleutians.  This  highlights  the  need  to  periodically  update  the 
traffic  model  used  for  Pid  to  match  the  observed  traffic  patterns.  Another  factor  influencing  traffic 
routes  are  local  laws  such  as  the  west  coast  tanker  exclusion  zone  that  forces  tankers  to  stay  well 
offshore  of  Vancouver  Island. 

The  best  way  to  outline  the  major  routes  (which  could  change  seasonally)  is  to  observe  a  map 
of  RMP  traffic  over  a  significant  period  of  time  and  highlight  the  most  dense  routes.  Ideally,  the 
period  of  time  should  be  long  enough  to  collect  data,  but  short  enough  to  observe  seasonal  traffic 
variances.  A  period  of  three  to  four  months  of  data  is  usually  sufficient  to  pull  out  the  major  routes. 
Once  the  routes  are  mapped,  then  it  is  straightforward  to  measure  intersection  of  the  routes  with 
the  predefined  surveillance  zones  used  for  patrolling.  The  distance  each  route  travels  through  each 
zone  is  measured,  and  used  as  an  input  for  the  calculation  of  Pid- 

Figure  4  shows  a  contrast  enhanced  map  from  an  open  source  image  obtained  on-line  from  the 
European  Commission  Global  Environment  Monitoring  website  [14].  The  authors  of  the  image 
describe  their  method  in  a  paper  [15].  The  image  was  generated  from  one  year  of  voluntary  reporting 
captured  in  the  World  Meteorological  Organization  Voluntary  Observing  Ships  Scheme  beginning 
in  October  2004. 


Figure  4:  Map  of  open  source  shipping  data  for  the  NW  Pacific  Ocean.  The  JTFP  outer,  middle, 
and  inner  surveillance  zones  are  shown  (from  left  to  right) 

4Unimak  Pass  is  a  strait  near  Unimak  Island  in  the  Aleutian  chain  of  islands  off  of  Alaska.  Traffic  from  Asia  following 
the  North  Pacific  great  circle  route  transits  through  this  pass. 

5The  Strait  of  Belle  Isle  is  between  Labrador  and  Newfoundland  and  connects  the  Gulf  of  St.  Lawrence  to  the  Atlantic 
Ocean. 


DRDC  CORA  TM  2009-037 


17 


From  this  data,  five  major  incoming  routes  arc  pointed  out  with  white  boxes  and  lines  in  Figure  4. 
Label  1  points  to  the  northern  shipping  routes  from  Alaska.  There  arc  two  sub  parts  to  this  route,  the 
1-a  line  points  to  the  modified  tanker  traffic  routes  which  arc  diverted  due  to  the  voluntary  tanker 
exclusion  zone.  The  1-b  line  arc  the  main  shipping  routes  from  Alaska,  particularly  Valdez  in  the 
north  east.  Label  2  points  out  the  northern  great  circle  route  that  is  squeezed  through  Unimak  Pass 
and  terminates  in  the  Strait  of  Juan  de  Fuca.  Label  3  is  the  southern  great  circle  route  which  is 
traffic  headed  to  the  United  States,  but  passes  through  the  JTFP  AOR.  Label  4  points  to  the  traffic 
from  Hawaii,  and  Label  5  shows  the  coastal  traffic. 

For  each  of  these  routes  shown,  the  distance  through  each  zone  is  measured  and  used  as  the  transit 
distance  for  the  Pm  equation  above.  For  example,  the  northern  great  circle  route  (Label  2)  transits 
approximately  260  nautical  miles  (nm)  through  the  outer  zone,  380  nm  through  the  middle  zone, 
and  520  nm  through  the  inner  zone.  The  transit  distances  for  each  route  through  the  surveillance 
zones  arc  calculated  in  a  similar  manner.  These  transit  distances  arc  then  used  to  estimate  the  typical 
transit  time  in  Equation  9.  Equations  15  and  16  arc  then  used  to  calculate  the  route  average  Pm  for 
each  zone. 

Note  that  the  methodology  for  defining  traffic  routes  described  here  is  based  off  of  open  source 
unclassified  data.  A  detailed  description  of  the  route  methodology  used  for  operations  will  be 
calculated  in  a  classified  DRDC  CORA  Technical  Note  [5], 
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3  Discussion 


Using  route  averaged  probability  of  identification  as  the  primary  metric  for  ISR  reporting  has  several 
advantages  and  limitations.  This  chapter  discusses  these,  as  well  as  possible  applications. 

3.1  Advantages 

A  major  role  of  maritime  ISR  activities  is  to  identify  vessels  on  the  ocean,  therefore,  measuring  the 
percent  of  vessels  that  arc  theoretically  identified  is  a  direct  measure  of  the  effect  of  ISR  activities 
and  their  contribution  to  building  the  RMP.  The  identification  of  vessels  in  the  RMP  is  the  first  step 
required  for  a  threat  assessment. 

Government  mandated  and  voluntary  Automatic  Identification  System  (AIS)  information  has  many 
implications  for  ISR  and  RMP  quality  [16].  Although  AIS  provides  information  on  the  identity  of 
vessels,  the  source  can  be  altered,  so  the  level  of  trust  in  the  data  source  comes  with  uncertainty. 
P/d  with  visual  identification  mitigates  the  risk  introduced  by  blindly  following  AIS  reported  infor¬ 
mation.  The  limitations  of  other  identification  methods,  such  as  AIS,  can  be  avoided  by  introducing 
caveats  to  the  P/d  calculation  such  as  visual  identification,  or  if  available,  multiple  sensor  fusion. 

The  chance  that  a  threat  to  Canadian  national  security  is  not  identified,  and  thereby  not  responded 
to,  is  of  great  concern.  A  worst  case  scenario  is  a  non-emitting  vessel  transiting  through  the  AOR 
with  hostile  intent.  Using  visual  P/d  as  the  primary  metric  to  assess  ISR  effect  directly  addresses 
this  threat  to  Canada  by  indicating  the  effectiveness  of  surveillance  in  identifying  the  non-emitting 
incoming  vessels.  The  P/d  metric  simplifies  the  task  of  identifying  ISR  vulnerabilities  to  observing 
a  P/d  map  of  the  AOR. 

3.2  Limitations 

The  only  accepted  way  to  detect  a  non-emitting  vessel  is  by  active  observation,  and  visual  identifica¬ 
tion  {P/d)  is  presently  the  only  sure  way  to  detect  and  identify  a  non-emitting  vessel.  Because  visual 
Pid  is  directed  to  addressing  the  worst  case  scenario,  using  P/d  as  the  sole  measure  for  ISR  effect 
can  have  the  effect  of  making  the  RMP  quality  appeal-  worse  than  it  may  actually  be  for  everyday 
intelligence  activities.  This  leads  to  a  risk  of  under-estimating  the  overall  surveillance  performance 
when  visual  identification  is  not  required  to  identify  the  threat. 

The  route  averaged  P/d  metric  alone  does  not  provide  a  complete  measure  of  surveillance  effec¬ 
tiveness.  The  quality  of  tracking  a  vessel  has  not  yet  been  considered,  and  the  development  of  a 
quality  of  tracking  metric  in  combination  with  the  P/d  metric  would  provide  a  more  complete  mea¬ 
sure  of  surveillance  effectiveness.  Also,  measuring  the  function  and  impact  of  intelligence  in  the 
threat  analysis  is  not  captured  in  this  metric.  Measuring  the  ability  to  produce  a  threat  assessment 
is  complicated  and  involves  many  components  including  identification,  tracking,  and  analysis. 

The  P/d  calculation  is  dependent  on  knowledge  of  traffic  patterns  in  the  surveillance  area.  However, 
the  actual  incoming  threat  may  not  follow  traditional  traffic  patterns.  This  limitation  is  mitigated  by 
the  fact  that  reporting  P/d  for  the  major  traffic  routes  maximizes  the  number  of  vessels  considered 
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Figure  5:  Effect  of  transit  time  for  random  and  regular  patrols  on  Pid  with  a  revisit  time  of  6  days. 


identified.  Also,  by  defining  many  routes,  the  typical  transit  distance  through  surveillance  zones  is 
averaged  to  somewhat  account  for  traffic  that  is  not  on  the  major  routes. 

The  Pid  calculation  is  dependent  on  knowledge  of  the  speed  of  an  expected  threat.  Figure  5  shows 
how  the  Pid  equations  depend  on  the  theoretical  speed  of  the  vessel  (shown  as  transit  time).  Chang¬ 
ing  the  value  of  the  transit  time  in  Equation  9  has  a  significant  impact.  Because  of  this  strong 
dependence,  the  reported  value  of  Pid  for  a  patrol  zone  should  also  be  associated  with  a  threat  speed 
to  indicate  that  it  is  only  accurate  for  a  specific  case.  As  long  as  the  threat  speed  is  not  extremely 
high  or  extremely  low,  the  Pid  value  will  be  a  close  approximation  of  the  chance  of  identifying  a 
threat  traveling  a  different  speed  than  expected. 

3.3  Applications 

There  are  several  applications  for  the  Pid  metric  in  planning,  reporting,  and  analysis.  Most  obvi¬ 
ously,  Pid  can  be  used  as  a  measure  of  operational  effect  of  ISR  activities.  In  this  way  it  could  report 
the  average  probability  over  a  weekly  or  quarterly  period.  It  could  also  be  used  as  a  quantitative 
measure  to  compare  different  ISR  activities,  platforms,  or  techniques.  Pid  could  also  be  used  in 
planning  to  set  a  goal  for  operational  achievement  [4]. 

For  ISR  planning,  having  a  base  measure  such  as  Pid  provides  a  means  to  quantitatively  define  what 
the  desired  ISR  effect  should  be.  For  example,  one  could  determine  a  desired  Pid  level  for  an  area, 
and  then  calculate  the  actual  value  resulting  from  the  implementation  of  the  ISR  plan.  This  could 
identify  a  capability  gap  or  an  area  for  improvement. 

It  could  also  be  possible  to  develop  an  instantaneous  geographically  distributed  value  of  Pid ■  This 
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would  require  knowing  the  actual  flight  times  and  defining  a  temporal  dispersion  or  decay  function 
of  the  Pid ■  The  temporal  function  would  define  how  the  instantaneous  Pm  would  move  along  a 
threat  course  over  time  and/or  whether  the  total  probability  would  begin  to  spread  out  or  decay  over 
time. 

Finally,  historic  records  of  Pm  values  can  provide  some  indication  of  risk.  By  comparing  the  levels 
of  P[[>  at  various  times  of  the  year,  the  relative  difference  in  risk  can  be  inferred.  Consider  the 
hypothetical  example  that  if  the  typical  Pid  in  a  zone  is  50%  in  the  winter  and  80%  in  the  summer, 
then  one  could  note  that  there  is  a  higher  risk  in  the  winter  that  a  non-emmitting  vessel  will  transit 
undetected.  This  could  allow  the  Pm  metric  to  contribute  to  a  larger  measure  of  risk.  The  concept 
behind  quantifying  risk  has  not  yet  been  developed,  and  is  currently  being  explored  by  Canada 
Command. 
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4  Summary 


An  initial  metric  for  measuring  the  effect  of  ISR  activities  has  been  presented.  The  averaged  route 
Pjd  metric  presents  a  clear,  easy  to  interpret,  quantitative  measure  of  the  ISR  activities  over  a  period 
of  time.  The  Pjd  metric  contributes  to  the  reporting  aspect  of  ISR  and  it  can  also  provide  guidance 
for  planning.  When  measuring  Pjd,  the  dependence  on  vessel  traffic  routes  and  speeds  must  be  kept 
in  mind. 

This  paper  provided  the  theoretical  foundations  for  the  Pm  metric.  It  is  one  of  four  papers,  each  of 
which  is  tailored  to  a  different  audience.  A  paper  on  adapting  the  metric  for  operations  is  described 
in  a  paper  by  Carson  [3].  The  implementation  of  the  metric  as  a  computer  program  is  described  in 
a  paper  by  Wind  and  Horn  [4].  A  classified  Technical  Note  outlines  the  calculated  results  for  traffic 
routes  for  the  JTFA  and  JTFP  AOR  [5]. 

Future  Work 

The  theoretical  basis  laid  out  in  this  paper  will  serve  as  a  starting  reference  for  future  papers  extend¬ 
ing  this  work.  A  technical  paper  on  the  operational  implementation  of  the  Pjd  metric  will  follow, 
along  with  a  paper  on  how  to  take  advantage  of  this  metric  for  the  refinement  and  development  of 
maritime  ISR  plans. 

The  next  step  in  advancing  the  utility  of  Pjd  is  to  create  a  fine-time  definition  that  will  enable 
approximate  real  time  Pjd  reporting.  The  availability  of  that  type  of  information  will  shorten  the 
Observe  Orient  Decide  Act  (OODA)  loop  by  significantly  improving  short-term  RMP  planning. 
This  capability  is  being  developed  by  the  JTFA  ORT.  Additionally,  the  development  of  a  quality 
of  tracking  metric  to  be  reported  in  parallel  to  the  Pjd  metric  will  significantly  improve  the  overall 
surveillance  effectiveness  measure.  JTFA  and  JTFP  ORTs  are  currently  working  on  this  additional 
tracking  metric  to  be  reported  in  parallel  to  Pjd- 

With  constant  addition  of  new  sensors  and  fusion  tools,  it  will  be  important  to  evolve  the  ISR 
effect  metrics  to  account  for  these  new  capabilities.  Although  the  Pjd  metric  presented  here  was 
for  visual  identification,  an  extension  of  the  metric  that  incorporates  an  information  trust  coefficient 
would  enable  Pjd  reporting  of  non-flight  ISR  activities  such  as  Synthetic  Aperture  Radar  (SAR)  data 
and  Long  Range  Identification  and  Tracking  (LRIT).  Work  towards  defining  empirical  coefficients 
through  data  collection  is  being  pursued  by  the  JTFP  ORT. 
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